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ABSTRACT 

Selected results from a study of compensatory reading 
programs sponsored by Title I and by other sources are presented and 
discussed.- Various phases of the study included a questionnaire 
survey of a . nationally representative sample of elementary schools, 
pre- and posttesting of students in grades two, four, and six of a 
subsample of the original group plus additional ^hools with unusual 
reading programs, an examination of summer programs in a subsample 
from the second phase, and visits by teams of observers to a selected 
group of schools that displayed a range of effectiveness. This 
summary is organi zed; around a series of questions, and draws on the 
results of each phase, as appropriate. General characteristics of 
compensatory programs are described, attributes and performance of 
compensatory and noncompensatory students . are compared, reading-test 
gains are analyzed, unusually effective programs and summer programs 
are described, and .results are compared with those from other 
studies.* (AA) 
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Contained in this summary* are selected results from a large. scale study of 
compensatory reading' programs funded primarily by Title I of tKe Elementary 
and Secondary Education Act, The study was initiated, by the Oiffice of 
Planning, Budgeting /and Evaluatlora (OPBE) of the U.S. Office of Education 
and was conducted by the Educational Testing Service and RMC Research 
Corporation. / , 

The study had three major objectives: 

.To obtain a detailed description of compensatory reading . - 
practices throughout the nation in grades 2, 4 and 6; 




To determine the possible relationships of such practices 
with student reading skill acquisition and their relative 
^,costs ; 

To obtain a detailed description of those practices that 
were found to be associated with unusual effectiveness. 

To accomplish these objectives the study was designed to be carried out 
in phases. The first phase involved a questionnaire survey cf a nationally 
representative sample of U.S. public elementary schools to obtain informa- 
tion on their regular and compensatory reading practices. The second phase 
involved pre and post-testing of all students in grades 2, 4 and 6 of a 



Prepared by the U.S. Of f ice of Education. • 



subsample of 'the original group of schools plus an additional set of. 
schools with noteworthy (unusual) reading pr'ograms .* A third phase of the 
study examined summer programs in a subsample of schools from the second 
phase, A fourth phase of the study entailed a series of^a visits by teams 
of observers to a selected group of schools that displayed a range of 
effectiveness to verify ongoing practices, suggest reasons for 
program effectiveness and obtain detailed information on those 
found to be unusually effective. The following summary is organized 
around a series of questions and answers drawing on the results from 

" each phase as appropriate. , ~ _ 

One of the first Issues to be addressed in the study was that of delineating 
what was meant by "compensatory" reading instruction. A very basic defini- 
tion was adopted: "any reading instruction provided to students because 
they are reading below their grade level:." When combined with other in- 
formation about the schools-and their students, the definition could be 
narrowed in a number of alternative x^ays _f or analysis pur-poses. 
The purpose of the first phase of the study was to obtain a nationally 
representative description of compensatory reading practices in 
the elementary grades and to select schools. for more intensive study 
during the subsequent academic year. Using the above definltipn of 
compensatory reading a nationally representative sample of 731 
elementary schools were selected on the basis of their socio-econdmic 
characteristics (comnunity income, percent minority and Title Ineligibility)* 
number of students, and geographic locales (regional and rural-urban' location) 
The principals and the teachers in grades 2, 4 and 6 of these schools were 

_asked_t6„provide-detailed-informatlon-on-their^^^ practices, training 

and experience, school characteristic? and. programs . 

* The. second phase was carried out during the 1S72-73 school year.- 



From the 5A3 schocls responding to the survey* the following results 
were obtained: 

90%. had some kind of compensatory reading instruction and 
70% received Title I funds; 

The dominant instructional approach was linguistic-^netic 
used by: 66, 54 and 33 percent of second, tourth and sixth 
grade teachers', respectively; ^ 

Only 5% of teachers did not use basal readers; 

. 20% of teachers reported tha,t_they had free choice of 

instructional materials; ^ . * 

. 25% reported that, they had no choice at all. 
. However, almost all supplemented with materials. they 
devised themselves; 

Compensatory reading was most often conduc ted d uring regular 
reading instruction times; ~~ 

Next most frequent V7as before or after school hours 
or during the summer; 
Compensatory reading practices changed more during the school 
year than did non-compensatory practices; 
- . The different apprcfacbes.; to compensatory reading could be reduced 
-to five basic descriptors along which they (viz. the approaches) 
- differed to different degrees-; — They were: - 



Twenty-one schools ~v7ere no longer in existence and there' was a slightly 
greater incidence of large city schools not responding. Seventy-six 
percent of the schools that could .respond actually did so. See Rubin, 
et al., 1973 and Trismen, et al/, 1975b. for details X^^e, results from 
the first phase. 



. Emphasis, on basic' reading activities 
. ■ Emphasis on use of " audio-visual materials ^ 
Emphasis on supplementary reading activities 
Emphas^is on instructional flexibility 
■ . • Emphasis on instruction during time 
released fro*m other school subjects. 
By grouping the programs by their similarity on these five 
descriptors 11 distinctly different types of approaches could, 
be readily identified. 

There were greater differences among schools in these approaches 
than there were differences among teachers within a school,' 
suggesting that the school (and its teachers) is the basic unit 

-- for -understanding differences among compensatory programs. 

On the basis of the returns to this survey, the following types of schools 
were studied intensively during the subsequent academic year (called the . 
second phase of the study): 
126 schools with compensatory reading programs funded by Title I 
80 schools with compensatory reading programs not funded by Titl^ I 

(called non- Title I schools). 
26 schools with no compensatory reading programs (called non-compensatory o 
' schools)*. . • 

■ . -^^ - . ■ ■ 
34 schools with noteworthy (unusual) compensatory reading programs 

(representing a. variety of funding sources but selected 

* purposively; 28 of these schools were Title I funded), 

266 Total* ■ 



* A -slightly greater proportion of non-Title I schools that were -requested 
to participate in Phase II actually did so (some 8 3%) compared with schools 
with Title T programs or without any compensatory reading program (some 

__73% each) (see Trismen, et al. , 197.5a, p.. 37). 



All students in grades 2, 4 and 6 of these schools were'^ administered' 
a set of standardized reading skill measures in the early Fall ani»^ 
again in the late Sprijig of . the regular; school year.* ' (Two versions 
of such tests, were given in order to encompass a range of 
development.) Included also 'was a specially 'developed instrummit ■ 
assessing a student's liking for reading^and positive feelings about 
himself as a reader. During the- school year information was 
collected on: each student's daily attendance in r^sading instruction; 
details on each teacher's instructional practices in reading as well 
as the nature and extent of their training and experience; individual 
studentf:lbiographical information such as their prior exposure to 
compensatory instruction, ethnic background, : participation in the 
free lunch program, etc.; the resources .utilized by the different 
instructional approaches and the standard costs of such resources. 
In addition the principal provided detailed information on the 
schools' policies and practices. 

The amount of resultant"" information encompasses some fifty-five 
• * ft 

thousand students, almost three thousand of their teachers and the 
principals of the 266 schools located In almost that many school 
districts. Analyses of such a voluminous amount of information can 
yield lengthy and complex results. For these the reader is referred 
to^the technical reports listed in the last pages of this summaify. 
Selected results will be organized, in what follows, around a basic 
set of questions pertaining to who gets compensatory ^ervices and 
how they might benefit from such assistance. 



* The tests were administered by personnel especially, hired and trained 
.for these purposes. Test scores collected especially for this study were 
not available in the early Fall to use as a basis for assigning students 
to compensatory or non-compensatory groups. 





I^^ThB^lrst question asks: 

.1. In! schools that offer ibmpensatory assistance In reading* 
. what proportion of the students arv grades 2, 4 and 6 receive 
such assistance and how do theseXprc portions va#y by source of 
funds? ' ' ^ - 

About 45, 40 and 37 percent of iibe students at grades 2, 4 and 6 
resp^ectlveiy, In such schools receive some form of special assistance 
In reading to compensate for their below grade level performance.* 
Comparable percentages, by "source of funds are: * - ' ' 



% Receiving Compensatory Assistance 
In Reading 



GRADE LEVEL: 



Funding Source* * 

Schools ^receiving Tfile I 
funds 



Schools receiving funds other 
than from Title I (Non-^Tltle I) 



48 



44 



44 



Average 
45 

35' 



41 .33 30 
These results show that large proportions o^ students - 
In these schools trecelve some form of compensatory 
I • assistance in reading with the proportion served being 
greatest at the lower grades and greatest In schools 
that receive Title I funds. 
The second question asks: 

2., How do students who receive compensatory assistance in 
reading differ from other students with regard to: their fall 
test performance; their liking for reading; their attributes and 
experiences; and, how do such results differ by the source of funds"? 



See Table 16B of Trlsmen, et al. , 1975a. 
1976l{^^^'^^"^^^^^ Revised Tables 17A-C of ' 



Trlsmen, et. al. , 
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(a) Student's fall test performance comparefi to national test 
norms ,^ . , 

The percentile rank for the typical student who received compensatory 
assistance in reading was 22 for each of grades 1^ U, and 6*. The 
percentile rank of the typical student who did not_ receive such 
assistance was 46. 48 ajid 53 for grades 2, A and 6, respectively. 
.Hence, their unassisted peers were near the national average in their 
level of reading skill development. . ' . 

' . ■ _ . . ■ • ."•'(■• 

There were some noteworthy differences among schools with funds from . 
'different sources as cai^be seen; from the following :^ ^ ' 



• * Obtained by converting the average MAT Total values for grades 2 and 
. 4 and STEP II total values for grade 6 in Table 23A (Trismen, et al. . ■ 
" 1975a:) to their. .percentile e,quivalents using the individual norms 

tables. Averages will tend /to-be closer to the national mean on 
■. . .. -^individual norms than they would on a distribution of group averages; 

Er|c 12 



Fall Percentile Ranks for Students Who 
... Do and Do Not Receive Compensatory 

Assistance In Reading* ■ 

Compensatory Students Non-Compensatory Students 

GRADE LEV1SL: _2_ .^5- -4- — ^ A _6 

Funding Source . - , . 

Schools thac receive Title I . ... 

funds'' . . 20- 20 22 42 46 50 

Schools receiving funds 
from sources other than 

Title I (Non-Title I) 24 24 24 48 54 56 

1 ^ ■ ; . ■ \ 

Non-Compensatory Schools - V 42 40 50 

• • . \ 

These figures show that compensatory students are about the same distance 
behind the national norm (viz. the 50th percentile) at the different grade 
leviels whereas the relative position of unassisted students (viz. non- . 
compensatory' students) varies in a progressively increasing manner with 
the ^ade level (the only excerption being non-compensatory schools at the 

fourth 'grade) . Students in non-Tit J^fe i schools rank uniformly higher 

• ►* ■ ' ■ ' ■ ^ . / • 

than do students in •Title I schools. The fact, th^t these.^dif f erences 

' ; ■■ _ ^. ^ — r 

(viz^ the difference -bfttween the compenpktory and non-compen^^tory 
students-* ranks) are small«r for students in Title I-, schools titan for t.;^ 
students in non-Title I schools is a maiiif estation of the greater con- 
^entration of lo\\' scoring, students in Title I 'schools. /t - 



* Obtained by converting the. average raw scores iii Revised Table 25A to 
their percentile equivalents using the individual norms tables (Trismen, 

/et al., 1976c). The MAT TOTAL was used for grades 2 and 4 and the 
STEP if -Reading /or grade 6. 



. Comparative analyses of schools with high and Tow 
concentrations of poverty level students showed 
that about 25% more of the students in high poverty 
schools would have qualified for compensatory services 
had they attended low poverty schools. This tended 

.to be so for schools within each source of^funds at 

mi' • 

each grade level.* 



(b) Their liking for reading and positive feelings about 
themselves as-readers .in the fall 
At the second grade, students who received compensatory assistance had 
slightly less favorable feelings about themselves as readers and their 
liking for reading than did students who did not receive such assistance. 
At the fourth and. sixth grades compensatory students had slightly more 
favorable attitudes than- did non-compensatory^ students. (These slightly 

ay 

more favorable attitudes could not be attributed to the compensatory 
assistance they received in prior years**) . 



See Tablg'24 of Trisnen/et al. , 1975a. Theoe results are based upon 
an early classificatlgp of schools before those with an "unknown", 
funding source were clarified and reclassified. For these latter 
see Tricmcin, et-al. , 1976c« , 

** See: Table 22 and Table 29 of Trismen, et ai., 1975a. 
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. Students in Title I and in non-Title I schools were about equal* I5 
the positiveness of their feelings [ viz. there were no appreciable 
differences (three percent or more of the variance accounted 
for) for students in Title I versus non-Title I schools], 
(c) Student *s attributes and experiences* 

About 56% of the recipients were boys 
. Compensatory assisted studetlts were about 2 months older 
than non-compensatory students. 

On the average 62 percent of the participants were white. 

By source of funds and grade level the results were: 

% White of Compensatory Assisted Students 

GRADE LEVEL: 2 4 6 Average 

Source of Funds ... 

Schools that receive Title I 

funds 61 . 59 59 60 

Schools that receive Non- 
Title I funds 72 70 65 69 . 

. There tended to .be a greater incidence of non-whites in 
separate instructional groupings an^ this incidence was 
greater than would be expected solely on the basis of 
depressed reading scores, [(However, non-whites may have 
additional kinds of educational disadvantageiu^mt which 
warrant such separate groupings (e.g. limited English- 
speaking ability.)] 



* The following statements will be qualified only when they do not hold 
for the different grade levels and funding sources. The figures cited 
were obtained from: sex, Table 14B; age. Table 20; ethnic background^ 
Tables 18A-C, 20A-F; compensatory assistance and free lunch, 16A-C' and 
17A-C; grade level status and compensatory assistance, Table A2; prior 
exposure, Table 31, of Trismen, et al., 1975a. Results for funding source 
comparisons can be -found in the corresponding numbered revised tables of 
Trismen, et al., 1976c. 
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. Of those students who received compensatory assistance In 

reading over one-half also participated In the free lunch 

program (54,. 60 and 56% for grades 2, 4 and 6 respectively). 

.•These results varied by source of funds as follows: 

% OF COMPENSATORY READING STUDENTS 
WHO PARTICIPATE IN FREE LUNCH PROGRAM 

SOURCE OF FUNDS 2 4 6 Average 

Title I funded schools 60 66 63 64 

Non-Title I funded schools 39 43 43 42 

In summary, these results show- that ai^icag students who receive compensatory 

asslstance~in--readlng-there--are-Jnoxe_^^ whites; and, 

free lunqh participants. In schools that receive Title I funds greater 

proportions of non-whites and free lunch participants are provided 

assistance than Is so In schools that do not receive Title I funds. 



, Slightly less than half of the students at each grade level 
(46% for grades 2 and 6, 48% for grade 4) participated in 
the free lunch program. 

..Of the free lunch participants, over one-half received' 

compensatory assistance in reading. 
.* These results varied by source of funds and their grade 

level differences were: ^ 

16 



' % of FREE LUNCH PARTICIPANTS WHO RECEIVE 

COMPENSATORY ASSISTANCE IN READING 

SOURCE OF FUNDS 2 ' Average 

Title I funded schools '57 53 _ . 55 55 

Non-Title I funded schoole 43 39 32 '38 

Ihe prciceding results show that a large proportion (about one-half) 
of students from poverty backgrounds (viz. those who qualify for the 
frc6 lunch program) receive compensatory assistance in reading and 
that this proportion tends to be greatest in schools that receive 
Title I funds. . 

^ ' . . Of those students who were one or~*inpre~~ire"arB~beiow ■ 

grade level (one-half at grade six, two-fifths at 
grade four and none at grade two*) 57% received 
compensatory assistance in reading at grade 6 and 68% 
" at grade 4. 

.. By funding source the results were: 

A B ■ 

% ONE OR MORE YEARS % OF A RECEIVING ; 
BELOW GRADE LEVEL - COMPENSATORY ASSETANCE 

SOURCE OF FUNDS GRADE : 4 \6 Average <i Average 

•Title I funded schools 45 56. ' 51 68 61 64 

Non^Title I funded 

schools 29 44' 37'- 68 51 60 . 



* In the fall of the second grade a student cannot be more than one year 
below grade level. (However, he can be very far behind his peers and 
very much in need of assistance) . 

Er|c . . 17 
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These results show that a large proportion (almost two-thirds )bf the 
students who are one or more years below grade level in reading receive 
compensatory assistance. The concentration of students below this level 
is greatest in schools that receive Title I funds. The proportion of 
students below' this level who receive such services is greater in Title 
I schools at the sixth grade. 



At grade six, 28% of the students were both one or more 
years below grade level* and free lunch participants; about 
sixty-one percent of them received compensatory assistance 

/ . ■•- ■ : . ■;■ \ ■ . . • ^ 

Jji_jeadijig_j£omp^r^^ 



and 70% respectively). 

.. By source of funds the figures are: 



% BOTH ONE OR MORE YEARS BELOW % .0F A RECEIVING 

GRADE LEVEL AND FREE LUNCH COMPENSATORY ASSIST/iNCE 

PARTICIPANTS . ■ . 

SOURCE OF FUNDS , GRADES : _6 Average A 6 Average 

Title I funded schools 31 34' 33 70 68 -69 

Non-Title I funded schools 13 21 17 70 49 60 

These results show that large proportions of students who are both 
from poverty 'backgrounds and are one or more years below grade level 
receive compensatory assistance in reading. The concentration of such 
students Is much greater in Title I schools and there is a tendency 
for proportionately more of them to be served at the sixth grade. 
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Ir*. the fall, students who were receiving compensatory assistance 
In. reading had varying degrees of exposure to compensatory 
assistance in prior. years as follows: 

% BY GRADE LEVEL 

1 A_ _6^_; ^ 

No prior compensatory 

assistance - 44 31 32 

• • One year or less of - 

compensa t ory-_ as sis tance ~ " ~~ 

in prior years 50 26 22 

More than one year of 
prior compensatory 

assistance 6 43 46_ lll:::^^: 

These results varied by funding source and grade level as follows*; 

% BY SOURCE OF FUNDS ' 



GRADE TWO 


TITLE 


I 


NON-TITLE : 


No prior assistance 


43 




46 , 


One year or less 


50 




49 


More .than one year , 


J 




5 


GRADE FOUR 


TITLE 


I 


NON-TITLE I 


No prior assistance 


32 




2a 


One year or less 


28 




22 


More thair one yiear 


40 




A9 


GRADE SIX 


TITLE 


I 


NON-TITLE I 


No prior assistance 


36 




26 


One year or less 


22 




22 


More than one year- 


42 




52 


* See Revised .Table 31 of Trlsmen, 


e t al . , 


1976c. 
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These results show that as one ascends the grade levels, proportionately 
more of the students being served have had prior assistance and 
proportionately fewer have had little (one year or less) or no prior 
assistance. This trend is more pronounced in non-Title I than in Title 
I schools. This difference may be due to the greater concentration of 
low scoring students in Title I schools and the Title I requirement to 
^t'^°^L^^y- 8tudents_each-year ' (i-.e .-as children" progfess~7~" 
through Title I schools there' may be a tendency to replace compensatory 

reading s'_udeiits who have achieved limited success by others whoi_aj:e_„: 

-more- ne'iuy)-. ' 

Our third question asks: 

3. Kow do the services that compensatory students receive differ 
from those received by other students ^-Tith regard to: the amount of 
reading instruction; and, the cost and nature of those services 
received?* 

^ (a) The student's expoeufe^to-^readin^ instruction during 
the school year**. 7 — 

. Overall, students who received compensatoty assistance in ^^^^ 
reading received exposure "to reading instruction on a greater 
number of days than did otfier students.' However, these 
results differed by classroom grouping practices and 
grade level as follows: 



* As before, the following statements will not be qualified with 
regard to source of funds unless differences were found, 

** See Tables 25A and B and 26A and B for. exposure rates and Table 
^35 for student movement, of Trismen, et al.^, 1975a. 
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Students who received compensatory asslstwce 

In classrooms In which there were also students 

who were not receiving compensatory assistance 

were exposed tc reading instruction on a greater . 

number of days (21 ,c 20 and 8 more dayo at grades 2, 

4 and 6 respectively) than were other students in the 

classroom, ' ^ „ [ ■_ 

Students who received their compensatory 

instruction in clajBsrgpms.^-wlth- 

compensatory students were exposed to such instruction 

on a slightly fever number of days than were 

non-compensatory students in other classrooms 

(1, 9 and 7 fewer days for grades 2, 4 and 6, 

respectively). 

... However, the instruction received in this 
latter arrangement may be more intensive 
(viz. lower pupil-teacher ratio, greater 
use of equipment and materials, etc.) for 
compensatory students when it is offered. 
Sixth grade students in non-Title I schools 
received about 11 hours more instruction in 
reading than did students in Title I schools 
(this refers to both compensatory iand non- 
compensatory students in such schools)-*. 



* See Revised Table 26A of Trismen, et al., 1976c. 
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Once it is decided !;hat a student is to receive compensatory 
asBistance in reading, he continues to receive such assistance 



^tKrougHout^Ti^'r^^^ year (viz. only one percent of 

all changes in statuR were from compensatory to'non- 

_ _ __cj)jmpen8a tor^ L . 

< - • ' . 

. Once it^s^d^cidedjt ■ 

^s^qfifive) compensatory assistance in reading, the anount of ■ ---^i--^^:.. 
reading instruction he receives does not vary with chis test 

score (Viz. within each of^he compensatory and -non-compenciatpry 
. categories there was virtually no relationship between a 

student's Fa reading test ..scores and his/her amount of" 

instruction)**. \ ^''^ • 

(b) The cost of services received***. ' 

(i) Total Standard Per Student Costs for Reading Associated 

» 

With Different Instructional Arrangements. 



* See Trismen, et al.', 1976c. . ^ . .. ' 

** See pp. 115-118 of. Trismen, et al. , 1975a. . ^ - ■ 

♦ ' . * ,*,*'. 

*.*.* These cost figures are not the usual kind of per-pupil expenditure 
anr^ consequently do not 'reflect all of the differences normally 
found in such ratios. They werf^ derived by obtaining detailed 
information, on the resources utilized in both regular and compensatory 
instruction* and then applying^ standard cost figures to these (e..g. a 
manufacturer's cost figure would be used for a teaching machine and 
a standard salary rate for a teacher of a given level of training and 
experience). For details see Dienemann, et al., 197A and Tables 16-18 
of Flynja, et 'al . , 1976. For th^ appropriate analyses see Trismen, 
et al., 1976c unless intiicated otherwise. 
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Compensatory Students 
\lxo Received Their 
Reading Instruction; 



'Average Number of 
Students Per Class 



Percent of All 
Students In Schools 
With Compensatory 
Programs 



Total Cost Per 
'Student In 
Dollars 



Iti classrooms that 
also contain non- 
^comp£naatory_atudents* 

In classrooms with 
only other compensatory 
v^udents ^ 



.52 



.$15.2.„ 



26 



23 



In-smalLp-speclal - reading" " "r 
groups with only pther 
compensatory students** 

In small, . special reading 
groups thatr.al6o contain 
non'-compensatory students**+ 



12 



2 



$199 



$664 



$580 



■ Non-Compensatory Students 
VJho Received Their Reading 
Instruction: 



In classrooms comprised 

only of other non-compensatory 

students " 28 

In classrooms of schools that 
• don't offer compensatory 
programs .. 27 
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$140 



$148 



* Tl^e cost model was not sensitive enough to pick up differences between com- '■ 
j>ensatory and non-compeiisatory students In the same classrooms. To do so 
detailed observations of resource utilisation- patterns within each classroom 
would have been required. Therefore, the cost figure is an average for all 
the children^ in the classroom. • 

** This amallXgroup instruction is in addition to their regular classroom 
' Instruct lon\ ' 

Non-compensatory students may be found in such classes because they attained 
this designation in the early Fall, but may have manifested a need for special 
assistance during the course of the school, year. > ' 

About 39 percent of the students in this category received compensatory 
assistance in reading. 
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The above figures do not represent differences within a single 
school district for. there' were almost as many districts as there 
were schools represented in the study (viz. nearly one school per 
district). They show thatP " — -r 



_.The_. average number of students enrolled in the different 
instructional groupings are very sljnnar~except~fbr the^^^^^- 
small, special reading groups which have almost two^thirds 
fewer students than the others. 

In schools that of fer- compensatory reading programs almost 
half of the students (some 52 percent) obtain their reading 
instruction in classrooms of mixed compensatory and non- 
cpmpensatory students arid of these students somewhat more 
than one-third (39%) receive some form of compensatory 
assistance in reading. 

• •• Slightly less than one-fourth (some 23 percent) 

of the students obtained their reading instruction 
in classes with only other compensatory students whil 
another twenty-one percent received their instruction 
in classes comprised only of other uon-compensatory 
students. 

.. About four percent of the students received their 



reading instruction In small, special reading groups. 
This instruction was in addition to "ttheir regular 
classroom instruction. 



Costsnof regular reading programs (non-compensatory) 
in schools that do and don't offer compensatory 
proc'cams are very similar ($140 versus $148), o . . 

Each category of compensatory student receives 
. ' 3%*, 34%, 349% and 292% resjpectively, more 

resources than do students, in schools that don't 
offer compenisatory programs. 

— Students in combined classes receive- about 

3 perceni:*^moxe_Tesources than do students in 
schools that don't offer compensatory programs 
($152 vs. $148). 

Compensatory students in separated classes 

. - ■ 5' 

receive about 42 percent more resources than 
their non-compensatory counterparts ($199 vs. 
$140). 

Students in small, special reading groups of 
mixed compe^nsatory and non-compensatory students 
receive 282% more resources than their com- 
pensatory counterparts in combined classes 
($580 vs. $152).. 



* Tnis ia an underestimate since the resource-cost model dia not 
differentiate between jgpmpensatory and non-compensatory students 
in the same classroom-rather a single dollar figure was used to* 
. represent the services for both categories of students. 

** Note the prior limitation on the totai cost f igure ^f or mixed classes 
of compensatory and non-compensatory students. 



^Students in small, special ^reading groups of 
only other, compensatory students recely* 
^234%. more resources than their compensatory 

counterparts in larger separate classes ($664 vW>v' 
■ $199). , 
Other analyses* showed that: .. . 

. Differences in costs of compensatory programs acrosts grade 
levels aren't appreciable whereas those for regular programs 
are (with costs for the latter being greater at the lower 
grades) . \ 

. For both corapfensatory and regular programs, cost differences 
. between schools were greater than differences among classrooms 
wifchiti schools. ' - ^' 

(ii)^ Per Student Reading Costs by Source of Funds** and Nature of 

Services Received. . " . . 

I. / ■ . ■ .• 

' ■ ■ 

Figures comparable to the preceding are given below for schools 

With"" compensatory reading programs ■ fuhded by Title I and for . 
schools with compensatory reading programs funded by sources other 
than Title I. 



Diem 



* See Dlenemann, et al. , 1974. 
** See Trismen, et al. , 1976c for these analyses. 



ft 



26 



1 1 










s 




















• 


■ w 


CO 










rH 


f 1 


1 


CO 




Q 




o> 


CO 


1 


Q 


1 




M M 


ri 
M 


r* 






CQ 




el 










VJ 


























ffi 










o ^ 










r ^ rl 




Is 


(• 00 




iH 


0) 


o 


in 


o 


<d 


iH 


o 


iH 


cs 


4J 






<o> 


<»> 


O 


fH 


o 






H 


H 


CO 







-22- 



•co- 



in 

m 

-co- 



CM 
00 

m 
■co- 



in 

CO 



m 
sr 





H 




CO 






4J 


0) 




s 


H 






•M 


09 




•H, 


iH 




EH 


0 


u 




0 


CO 






C 


U 


H 






TV 




of 






4J 


H 


CO 


a 




iH 


0) 


0) 


0 


o 


fH 


0 


M 


4J 


4: 


0) 


•H 


a 




H 


CO 



CO 

m 



CM 

m 



CM 



cn 



CM 



co' 












g 










H 








0) 






:) 






4J 


fH 






CO 


4J 


CO 






•H 


fH 








0 




0 




0 


5 CM 




a 


















CO 










a u 












.fl 








0) 








H 


CO 




to 








<0 


0) 


.0 




M . 


fH 


0 




0) 


4J 


^ 






•H 


.0 






H 


CO 





0) 

I; 

0) 

a 

O .(3 
CO Ci 

Si! 

CO 

3 a . 

*J H 

CO 

60 

&5 

O 

CO 01 

2 ^ 

S M 

g.jg 



NO 

CM 



CM 



CO 



CM 



EKLC 




2^ 



" d 

CI ^ 

CO g 
CQ 01 

i ^ 



CM 



C30 
CM 



M-l CO 
O 4J 

a 

fH 

o u 

CO 

(0 ^ 
'H O 
U U 
Pt (0 

S CQ 



CO 

B 
o 
o 

M 
CO 
CO 

02 
a 



a 
I 

a 

o 
a 

01 
H O 



60 

a 

•H 

a 
:) 
o 



These figures show that; . . 

. The size of the Instructions! groups (viz. average humber. of 
students* per class) are. nearly Identical for Title I and for 
non-Title I schools except for small, special reading groups 

which are very sllghtlv larger for Title I schools. ^ . 

^ ■ (- 

.. The percent of students served In the mixed classroom settings 
(viz; classrooms that contain both compensatory and non-compen- . 
satory students) is very similar for the .two categories of 
. schools but the. percent served by the remaining Instructional 
groupings differs as 'follows: 

• • Title I schools serve proportionately more of their 

students in separate classrooms (viz. in classrooms with 
only other compensatory students) and in 'small special 
reading groups than do non-Title I schools (6% versus 2% 
for the latter). > 
Non-Title I schools serve proportionately more of their 
students in classrooms comprised only of other non-com- 
\. p^aisatory students than do Title I schools* 

. Other analyses shqwed that of all the students in the study whose 
school had a compensatory reading program 62% were in Titlfe I 

■ • ■ /■ ■ ■• . * 

/ funded schools and 38% were in non-Title I schools*. 
/ * The total student costs do not differ substantially among the 
two categories of. schools with: 



* See Trismen, et al., 1976c. 
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• • ' Title I funded schools spending slightly more In 
. ^ absolute amountB (except for one category of small. 

special reading groups) than non-Title I funded 
schools. 

. . Non-Title I schools tending to spend proportionately 
more on their different categories of compensatory 
students than do Title I funded schools (viz.- relative 
to the dollar amount each spends oil claesrooms coinpri$ed 
only of other non-compensatory students) . 
. These Title I non-Title I cost values were not: consistent in the 
direction of their differences across the grade levels, however, 
the extent to which one exceeded the other was never very large*. 
Other analyses showed that; 

. Proportionately more Title I schools had programs which could be 
readily and zpeaningfully describied than did non-Title I schools** 

. More programs in non-Title I schools tended to deemphasize 
basiC; reading activities and put more exfip)uLsis on the use of 
audio-visual equipment and materials and indicated less instruct- 
tional flexibility than did programs in. Title I schools';*** 



* See Trismen,,et al., 1976c. . 

** Viz. More of them could be categorized in the typology of programs 
-that was developed. See Cluster 11 of Revised Table 40 of Trismen, 
-et al. , 1976c. 

*** See Clusters 2a and 4a of Revised Table "40 of Trismen, et al. 1976c. 
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These results. show that there was very little difference In class size 
for the different instructional groupings and funding sources except 
for- the small special reading groups which were almost two-thir^8 smaller 
than the other groupings. Compensatory assisted students received 
a greater level of resources tahn did non- 
compensatory students with this difference being most pronounced for 
small, special reading groups. Title I funded schools served more of 
their Students In such small, special reading groups than did non- 
Title I schools- (6% versus 2%). Overall, Title I funded schools tended 
to spend slightly more in absolute amounts on their compensatory students 
-than did non-Title I funded schools. However, non^Title I schools tended 
to spend proportionately more on their compensatory students relative to 
their non-compennatory students than did Title I funded schools. Finally, 
proportionately more Title I programs could be readily and meaningfully 
described than cdUld non-Title I programs. 



fourth question asks: 

' A. How do students who' receive compensatory assistance in 
reading benefit from these services when compared to other students 
with regard to: their test performance; their liking for reading; 
and, the cost of such services and their source of funds? 
To gauge how students. who received compensatory assistance in reading 
might have benefitted from their experiences, their reading test 
performance. can be compared with other students in the study who were 
not so. assisted and with national norms. Each of these comparisons is 
discussed in turn.. 

Befor^- discussing the results of these comparisons, however, it might 
be well to dwell on what they might be expected to show. ^ One important 
source of information about the performance of disadvantaged (minority) 
students pomes from the Equal Educational Opportunities Survey. Con- 
ducted in 1965 at about the time bf enactment' of Title I; of ESEA but ~ 
before the impact of funds resulting from it could begin to be felt, ' 
this national study showed that disadvantaged (minority) students fell, 
increasingly further behind their-more advantaged (non-minorityO peers 
in their performance on measures of* reading and mathematj.es at the higher 
^rade levels (Coleman, et al. , 1966)*. In the absence of Title I similar 
results might be anticipated--one may ask therefore of the extent to which 

_ compensatory programs have, in the aggregate, arrested., this decline. 

— — ' — ■ \ - . 

* These disparities were even more pronounced when students were 

separated into different categories of economic background <Okada, 
. et al., 1969)." Although the use of grade level equivalent scores 
..made this decline apj^ear worse than it actually was, a decline was 
also evident in a more acceptable metric (Mayeske, et.al; 1973a, 
page 115; 1975, page 48). (Such results were attenuated somewhat' 
at the lower grade le^^els due to problems associated with identifying 
young children's economic and ethnic backgrounds.) More recent 
evidence of a percentile decline comes from the Emergency School Aid 
Act Evaluation which showed that children in grades 3, A and 5 of a . 
nationally representativ^e sample of minority isolated schools (50% 
or more non-white) performed- at: the 23rd, 18th and 19th percentiles 
.respectively, on national test norms for reading achievement in the 
Spring of 1973 (Ozenne, D. G., et al., 197A). 31 ' 



insert to Footnote, Page 26 of the Technical Suirmiary 



Finally, examination of- trends for non-Follow Through students (those 
students who were selected for comparison purposes in order to gauge 
the growth of students receiving the Follow Through models) showed that 
they manifested a percentile decline and this decline was most 
pronounced for poor, central city, minority students. (Kaskowitz and 
Norwood, .1976) . 

Ref:'" Kaskowitz, D. and C. Norwood, 1976, A Study of .the Norm Referenced 

Procedure as Applied to the Evaluation of Project Information Packages 
Menlo Park, Calif: Stanford 'Research Institute. ^ 
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(a) The test performance of compensatory reading students compared 
with other students In the study. Our first comparison focuses on the 
relative improvement of different ""categories of students In terms of 
t^e percent of | Items completed correctly on all the reading skill tests 
-used In the Fall and again In the Spring,* To Illustra te . trends In the 
data and to simplify the presentation, we shall, express the number of 
iters that compensatory students complete successfully. as a percent of 
the items that wer^ successfully completed by students who did not re-, 
ceive such assist^ance. For example^ when we, cite a figure of 71 for 
second graders. in the Fall, we mean thaf 71 percent of the items success- 
fully completed by non-compensatory second graders were also successfully * 
completed by compensatory second graders.** A corresponding figure of 
85 for the Spring results means that the performance of compensatory ' 
assisted studeii'ts has mov.ed closer to that of students not so assisted 
in. terms of the number of items they can successfully complete. . If the- 
resultant figure were 100 it would . indicate that the average number of items 
correctly completed were the same for both groups.*** - 

* For reasbns that are, abundantly illustrated in Appendix A of this.- 
.* Nummary, grade level equivalent scores systematically distort test 
results in ways that' are detrimental to the judgment of student 
growth and project success. They should never be used in any kind 
of evaluation. * : 

** These ratios are obtained by dividing. the average number of items 
correct for compensatory students by the average number: correct for 
■ non-compensatory students (see Revised "Tables 23A, "and..B of Trlsmen, • 

■ et al., 1976.c). Leading decimal, points have been omitted.. 

. *** It should be noted that during this time period all students showed ■. 
. considerable growth* in terms of the number of reading skill items . 

they could successfully complete. For example, at the second grade. . 
in the F&ll the average student could complete about 49 items 
correctly and in the Spring 65 items. (see Mat Total, in Table 12 of 
Trlsmen, et al. ,' 1975a) . All "percent correct rates'Vare /computed 

■ on the Total score for Metropolitan Achievement Test. These 
"percent correct -rates" were merely computed for summary purposies 
using the Metropolitan results since- the Metropolitan results allowed 
for a comparison across the different analytic techniques (e.g.. » 
percentiles versus percent correct rates, etc.).-. The actual statisti- 
cal analyses were based on raw and standard test scores. For these- 

. ' see Tables: 23A and B; and Appendix C, of Trismen, et al. , 1975a. ■ 



These ''percent correct rates", for compeneatory students 
relative to students in schools without compensatory programs 
were: . " 

GRADES FALL SPRING SHIFT 

2 76 88 .• +12 

4 70 " 75 +5 

. 6 ' 78 83 +5 

"Percent correct rates" for compensatory students relative to 
non-compensatory students In all schools with compensatory 
programs were: ' 

GRADES FALL SPRING .SHIFT- . 

■ ■ ■ ■ ■ ■ . — ^ 

. • 2 71 85 ' +14 

A .65 . . 71 +'6 • - 

6 76 81 +;5 

; . These latter results "varied by source of funds as 

foll^ows:* ... 

GRADES 

Schools That Re-r . — ? — — 



celve Funds From: ^ALL SPRING SHIFT FALL SPRING SHIFT FALL SPRING SHIFT 
Title I 71 £j5 14 65 71 . 6 ' 75 80 5 . 

Non-Title I 69 85 16 . 64 72 8 77 . 81. 4 

These results show that compensatory students are further behind non- 
^ • compensatory students (In either their own schools dr in schools, that 
do not have such programs) in the Fall than in the Spring. , The extent 
of this "catch-up" is most pronounced at the second .grade Non-Title I 



* See the MAT Totals for Revised Tables 23A and B of Trismen, et al., 1976 
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schools show a slightly greater shift than do Title I schools, ' ^ • 
except at the sixth grade (it may be recalled that students in \ 
Title I schools ranked lower on the test in the'Fall than did \ 
students/in non-Title I schools) ; However, these increments are 
not appreciably different (viz. they do not account, for three percent 
or more. of the variance) for the different funding categories (see 
Trismen, et al., 1976c, Table A). 

. Among students who were receiving compensatory . assistance 
. . Those who had such assistance in prior years tended to 

rank lower on their fall reading test scores than 

. . * ^ ~^ — ■ 

students who did not have such prior asslistance and to 

■ \ - ' ' ■ . • " • 

gain an amount equal to or slightly -less than those who 

had never had such prior assistaiice.* 

■ ^ /' . ■ . . ■ 

The reader should be apprised of the fact that these results (viz. ^ 
^compensatory students being closer to non-compensatory students in 
the Spring than in the Fall) tend to be dependent upon the analytic ' 
method employed. / - 

Six different analytic techniques were employed. They were: 

1. Raw Scores ; for all students in the study these comparisons 
involved the absolute number of test items attained correctly in the 
Fall and in the Spring. absolute numbers are examined, as 

in Appendixi^A cJf this Summary, or their ratio's are taken, as they are 
^"^.r coifiparisons show that compensatory students tend to be closer 
to non-compensatory students in the Spring than in. the Fall. 



See Table 30 of Trismen, et al., 1975a. 
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2* Deviation Scores : in these comparisonff the average number 
of items correct for compensatory students is subtracted from that, 
of non-compensatory, students and this "difference is divided by the 
total variability, of all students in the study at that, point' in time 
(i^e.: the standard deviation). When such deviation scores are com-* 
puted for the Fall and Spring test scores, as they ate in ApfJendix B 
of. this Summary, they show that compensatory students are about as 
close or slightly tloser to non-compensatory students in the Spring than 
in the Fall. 'i--- 

3, Encoding of Compensatory-^Non-Compensatory Status : these 
.comparisons student compensatory-non-compensatory status is encoded as a 

quantitative variable which is then correlatied yith test scores in the 

Fall arid again in the. Spring. When such a correlation diminishes in 

... * 't ■ . " 

magnitude from Fall to Spring it quantitatively expresses the extent , 
to which the; groups have moved closer together . These kinds of analyses 
showed that compensatory-non-compensatory student test score^^^IfTererices 
tended to either stay the same or diminish somewhat from Fall to Spring 
-depending on the sub-test involved (see Appendix C of .Trismen, et al. , 
1975a). 

4. National Norms Deviation Scores : in the preceding analyses * 
-the magnitude of average differences between_categories of students . 

in the study were compare^\to^ each other ^ In the. current analyses, 
the. average performance of different categories of students in the 
study are compared to that of the^natlonal average student as described 
in test publishers norms. In these \nalyses the difference between each 
category of student: («.g.'' compen sat or y\r non-compensatory) 
and the national average is divided by the Xotal student 

36 
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dispersion (viz. standard deviation). Both Fall and Spring norms were" 
\ available only at the fourth grade. The results of these analyses, 
\ given in Appendix G of,. 'this Summary, showed that compensatory students 
tended to be closer to the national average in the Spring than in the 
Fall^s^ith the extent of this "ca'i^ch-up" depending on the .subtest involved. • 

* ^* -Raw Gains Analyses ; In these analyses the algebraic difference 
between a Btudent's. test score iVthe Spxlng and in the Fall is used 
as a variable which is related to. other variables (e.g. compensatory ' - 
versus aon-compensatory status) to see if they help ^ to explain why some 
differences are larger than others. ~ • * _r 

For these analyses the preponderance of the comparisons for the different . ;. ,'> 
tests and subtests either showjed no differences of practical significance 
between compensatory* and non-compensatory students (viz. the group 
differences accounted for. less than three percent of the variation) or 
showed differences favoring the gains of compensatory students (73% 
favoring compensatory Versus 27% for non-compensatory, see Table A of. 
Appendix C of Trisiaen, et al. , 197Sa*) . 

6* Residual Gains Analyses ; I'n^sidual gains analyses an expected ' 
Sirring test score is obtained for each instructional group. based upon it»s Fall 



*These percentages were computed from the following three comparisons 
in Table A; (1) All CR versus NCR in NCR schools; (2) Cr" combined 
•versus NCR combined both in CR schools ; and, (3)' CR separate versus 
NCR separate both in CR schools. Most of the differences of practical 
significance favoring the "gains" . of coirpensatory students came from 
the second comparison (11 of 14). • 



EKLC 



37 



test score and^-then, the extent to which an Instruct loTial group's actual 
Spring --test^'cofe exceeds this expected score Is obtained. Thl§ residual 
gain, score** (expected Spring score minus actual Spring score) is 
then related to other variables, such as compensatory-non-compensatory 
status, to see if they help to explain the magnitude of the residual 
gains. There were three kinds of comparisions of particular importance.* 
They were a comparison of the res^Ldual gain of: (l) all compensatory 
students with all non-compensatory studeats; (2) all compensatory students 
versur all student:s in schools that^did not offer compensatory programs**; 
and, (3) all compensatory students versus all non-compensatory students 
in their same schools. Results' for the first comparison showed that 
on only two of the seventeen test scores analyzed was there a difference., 
of any practical significance (viz., accounted ,for three percent or more 
of the total variance) and in these cases' (second" grade Metropolitan 
stories and Coop scales) non-compensatory studl^nts'^ained slightly 
more tlian compensatory students. For the remain?&, compensatory. . 
Students tjained to' about- the same extent as^did non-compensatory students. 
In the second comparison, compensatory studente gained to the same extent 
as did non-compensatory students for;14 of the 17 scales. For 'the 
remainder, the groups were not comparable due to their shaving .different- 
regression curves (at the sixth grade only). In the last comparison, 
there were no practical differences between the gains^^of compensatory 
and non-compensatory students for .15 of the 17 scales. Of the remainder, 

* , See Table 28 of Trismen, et al., 1975a.^ These analyses use^^'Fall test 
score plus its square to obtain an expected Spring score, l 

• . ■ * ' . " 

** It may be seen in the next section that the test scores "of students 
in these latter schools were higher than those of compensatory 
students in the former schools. 
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one showed differences favoring somewhat the gains of non-compensatory 

J ■ - ' *■ ■ ■ ■ i: ' ' - ■ 

students (second grade Metropolitan stories)- while for the other scale 

the groups ^were not comparable due to non-parallel rcgrefi.:i5ioa curves. ' 

For an overwhelming prep pnderance of^ these reading scales 'then (some 

86 percent overall) compensatory students grow at the same rate as. do 

non-compensatory studentB (viz. they do not fall further behind). A 

number of other analyses involvitig. classroom grouping practices showed 

that compensatory versus non-compensatory comparisons eithit-did ifot 

show any practical differences in. their gain (the one-exception being : 

the Coop , scale at the second grade) or that the regression curves for 

the two groups were not comp^arable (some 44 ^percent overall). These " ^ 

residual^ gain analyses then, support the ^s^sertion that compensatory 

students \do not fall further behind non-compensatory students. However, 

■ ■ ■ ■ 

they do not support the notion of their being closer together in the 

"#«■■' 

Spring than in the Fall. 

• ' ■ ■■ . ■ ■ . • ^ 

In sunmiary, all of these analyses .support the assertion that compensatory \ 

students tend not to fall further behind non-compensatory students during 

the academic year. Further, results for ;the first five analytic techniques 

tend to show that, ih going from Fall. to Spring, compensatory students 

improve their reading test scores relative to their non-compensatory 



counterparts. However, .since the residual gain analyses do not support 
this notion this latter assertion cannot be made unequivocally. To inaUe 
this assertion unequivocally one must first refute the notion that this 
"apparent movement" is not brought about merely by the tendency of a' 
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more extremej group to move closer to Its population value-^on a 
second testing occasion than would a less extreme group (the' / 
well-known "regression to the mean effect"). Although not 
completely refutable, such "regression effects" should be 

■ 4.'. 

minimal since the test scores collected as part of this study 
were not available for the use of local school personnel in 
assigningr 'Students to a compensatory or a' non-compensatory ^ 




group** In a^ldition,* all MtC ^t^sts \ s9d displayed a high degree 
-of internal consistency (see Table 1TS)F Trismen, et. al, 1975a) ' 

so ^hat extreme 3Core8' would be less likely to produce a 

/'regression effect?'^ As a consequence we shall regard this 
* "movemeSnt" as suggestive of the. upper^limit on the kinds of 

"gains" that might actually be occurring- ' ' 



(b) Reading test performance of compensatory reading students 
compared with national percentile test norms. Our next comparison 
focuses on fourth grade students and how they fared compared to 
national norms. Only fourth grade students are analyzed because 
both Fall and. Spring norms wer e not available, for the tests oused 
at the other grade levels.* - . 



.* Viz. these decisions were made on some basis other' than the test 
scores used for this study. . 



**. All. comparisons are based on the Metropolitan Achievement Test 
and were obtained by converting the average raw score to its 
percentile equivalent using the individual norms' tatjles. (See 
Reversed Tables 23A and B of Trismen, et al.,- 1976c). 
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c. The typical student who received compensatory assistance 
In reading at the fourth grade, maintained a 22nd per- 
• . centile standing from the early fall to the late spring 
'on the total reading scale. 
- . . .. A student who started at the 22nd percentile in 

the Fall but made no progreES during the school 
year (i.e., kept his 'same raw score) would have 
^ . dropped to the 12th percentile in the Spring,* 

.. For the same corresponding time period^ his/his 

typical advantaged schoolmate, who did not receive 
such assistance, moved from the 48th percentile 
of the national norm in the fall to the 50th per- 
centile in the spring—an increase of two percfentile. 
■■\. - ■ ■ ■' ' 

o ranks, while students in schools that did not offer 

compensatory assistance in reading liibved from the 
40th to the 44tfi percentile— an increase of four 
percentile ranks.** 
Iir tabular form these results- were : ' 

READING TOTAL SCALE / 
Student Category F/ill ' S pring 

Compensatory • , 22 . 22 

JJon-Compensatory in . 

non-compensatory schools 40 44 

Non-compensatory in 

compensatory schools 48 ' . 50 



* This is an example included from the test norms for comparison 
purposes. ' 

** Similar analyses based on deviation scores shew compensatory students 
to be about the. same distance behind or closer to the national average 
in the- Spring than in the Fall (see Appendix C of this report). • 
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• By subtests the results were: 

- For WORD KNOWLEDGE 

Student Cateogry Fall 

Compensatory <' 22 

Non-Compensatory in 
non-compensatory schools 40 

. Non-compensatbry in 

cbmpensatory schools 46. 

. . For READING ' 

Studept Category Fall 

Coxnpensatory . 22 - 

Non-compensatoiry in . 
non-compensatory schools 44 

Non-compensatory in 
compensatory schools 50 

These results show that for Wor^ Knowledge* compensatory students 

drop slightly from Fall to Spring while non^-compensatory students 

either stay the same or increase slightly. For the Reading scale 

both compensatory students and studeats In schools that don't have 

compensatory programs increase by 4 to 6 percentile tanks while the 

remaining non-compensatory students Increase by 2 percentile ranks. 

. By source of funds the results were: 



Spring 
2.0 

40 

48 

Spring 
26 

50 

52 



; ^ PERCENTILE RANKS FOR TOTAL. READING 

T?YPE OF STUDENT 

/ COMPENSATORY ASSISTED NON-COMPENSATORY ASSISTED 

Type of School Fall Spring Fall Spring 

Title I 20' 20 46 48 

Non-Title I ^ 24 24 54 gg 

Non-Compensatory - -r 40 

These results show that \ compensatory students - regardless of the source of 
funds, jnaln tain their same relative status from Fall to Spring.. . These = 
contrast with the results for non-compensatory students who show an 
Increase in their percentile rank with this increase being greatest in 
non-Title I schools and leiast in. Title I schools. However, there 
marked differences among the separate scales as can be seen from the 
following: 



PERCENTILE RANKS FOR WORD KNOVJLEDGE 



Type of School 
Title I. 
Non-Title I 
Non-Compensatory 



TYPE OF STUDENT 



COMPENSATORY ASSISTED 



Fall 
22 
24 



Spring 
18 
24 



NON-COMPENSATORY ASSISTED 



Fall 
42 
54 
40 



\ 



''■Spring 
!'44 
56 
40 



V 



43 




Type of School 
Title I 
Non-Title I 
Non-Compensatory 
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PERCEHTILE RANKS FOR READING 
TYPE OF STUDENT 
COMPENSATORY ASSISTED 
Fall " Spring 

20 22 
22 28 



NON-COMPENSATORY ASSISTED 
Fall Sprinjg 
48 50 
56 60 
44 50 . 



^ese results . shov/ ,«:hat , f or.: the. Wordi Knowledge sub-scale compensatory 
Students in Title I schools slip somewhat from Fall to Spring while 
thoiae in non-Title I schools maintain their same relative rank. . Non- 
cbmpensatory students in schools that don't have compensatory programs 
maintain their same relative status f rom= Fall to Spring whereas other 
categories of non-compensatory students- increase their percentile rank 
slightly. For the Reading sub-scale all categories of students increase 
their standing from Fall to Spring" with this Increase being greatest 
in non-Title I schools or in schools that don'tXpffer compensatory 



assistance. 



1 

1 
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Overall then/these percentile analyses show that the 
concentration of low scoring students is greatest in.; 
Titl^ I schools with the result that their compensatory 
and non-compensatory students rank lower than their • 
counterparts in non-Title I schools. Similarly, non- ' 
compensatory students in Title I schools lie closer to 
the rank of students .in non-compensatory schools- than 
to such students in non-Title I schools! With one 
exception" (Word Knowledge for compensatory students 
in Title I schools) all categories of students either 
maintain- their same relative rank or advance slightly 
with these advances tending to be greater in non-Title 
. I schools. UndouTjtedly the greater concentration of 
pading problems i*ti Title I schools as well as the fact ■ 
that they provide compensatory assistance to lower 
scoring (viz. more needy) students have an effect on the 
advances they are able to manifest. The differential 
gains, for the Reading versus the Word Knowledge scales 
may reflect a greater emphasis given to the former at the 
fourth grade. Finally, one wonders if the gains manifested 
by non-compensatory student's in schools that have compensatory 
programs might be attributable, in part, to the presence of 
such a program (viz. their performance is not held back by the 
slower students as it would likely be if all students were . 
receiving the same amount of instruction). 



(c) Student's liking for reading aetlvltles and pociltlye. feelings 

. * • ■ ■ . . • \ ■• 

.about themselves as readers. / 

. . ■ ♦ ■ ■ " • / ' ■ • ■: , • 

Students In this study can be compared with one. another as veil 
as with those In an earlier study with regard to their affective ^ 
growth.. Each type of comparison Is discussed In turn. 

From earlier discussions It may be recalled that the affective Instru- 
ment allowed a student to Indicate how positively he felt about 
different reading activities and about himself as a reader. In order 
to Illustrate the relative status of compensato:s:y and non-compensatory 
students on this measure we have computed ratio's similar to those 
In a preceedlng section. The numerator of this ratio represents a 
rescallng of the number of positive choices made by compensatory 
students and the denominator, a rescallng of the.niomber of positive 
choices made by non-compensatory students.' When the ratio Is^ess 
than one It Indicates that compensatory students made fewer positive 

choices than non-compensatory students; equal to bne*-.-that the groups 

'■ ' , 

are about equal; and, greater 'than one— that compensatory students had 
more positive choices than did non-compensatory students.^ 

. When compensatory students were compared with students in 

schools that did vnot have compensatory programs the fall and 

spring ratios were: 



* The following ratios are obtained from Table 28 of Trlemen, et'al., 
1975a by dividing the respective fall or spring values for com- 
pensatory stu4ents by that of non-compensatory students. For the . 
fourth and sixth grades a constant of one was added to each mean 
before division. The values used are for the comparisons: "All 
CR vs. NCR schools"; and, "All CR vs. All NCR, both in CR schools". 
The statistical analyses are not based on. the ratios. 



Gr&de 


r axx 


Spring; 


Shift 


2 


' .90 




.08 


4 


1.53 


1.66 1 


.13 


6 


1.68 


1.64 


-.04 


. When 


compensatory 


students were compared with non 


, students in their 


same schools the 


ratios were: 


Grade 


Fall 


Spring 


Shift 


2 


; .90 


.95 


.-05 


: A 


1.74 


. 2.03 


.29 


6 


1.82 


1.84 


.02 



These results tended to differ by the particular • 

comparisons being made and within these by the 

analytic technique employed and the grade 

level involved. However, for a preponderance 

of the comparisons there were either no differences 

in the growth of the two groups or differences 

sjLightly favoring compensatory students*. 

..• Students in Title I and non-Title I school^ grew 

more favorable about themselves to about the same 

extent at each grade level (there were, not - 

appreciable differences ^bng them)**. 

* ^^^^"flidual gain (covarlTncG) analyses in Table 28 of Trismen. et al. 
1975a the results were about one-third no dif f erencc . and .one-third slightly 
in favor of compensatory student gains. , For the unconditional analyse! In 
Appendix C, the gain score analyses in Table A showed no differences 
while those in Table B which/encoded compensatory -non-compensat«Py 
status as a variable, showed either no change in status from Fall to 
Spring or^ a shift.in the status of compensatory students closer to 
or surpassing that of non-compensatory students'; 

** Viz. differences that accounted for three percent or more of the variance 
(see Trismen, et. al, 1976c, Table A) . 



• Among students in the fourth and sixth grades who were 
receiving compensatory assistance, those who had such 
assistance in.f prior years tendedx^to rank about the same 
in the fall as those who. did not have any prior assistance. 
However, those who had received prior assistance became 
more favorable in their attitudes to a greater extent 
during the course of the year than did those who had 
not had any prior asaistance*. 
In a precediag section results from the Equality of Educational Opportunity^ 
Survey (Coleman, et .al., 1966) were ufeiad to give an approximate indication 
of what the achievement status of disadvantaged (r.inority) students might 
have been .prior to the initiation of Title. I. ij^g same study also , showed 
that disadvantaged (minority) students became progressively Bwre fatalistic 
about their ability to enhance their life circumstances through the avenue 
of education^. Although the current study did not measure a student* s 
sense of fatalism it did deal with Its antecedents insofar as they are 
rooted in his reading experiences. The results of the current study 
show that compensatory students become incrieasingly more favorable 
towards themselves as readers and in their liking for reading activitfes 
and Improve more in those attitudes than do nbn-compansatory students 
(within each of grades 4 and 6 only). As a result they come to equal 

or surpass/ their peers in this regard by the close of the academic year. 



* See Table 29 of Trismen, et al., 1975a. 

** See especially, Mayeske, et al, » 1973b, page 60. These were trends 
: in the grade level averages* 



(d) The cost of such^ services and their source of funds. ; ^ 
Results in an earlier section showed that the most educationally needy 
students, as evidenced by their depresssed reading scores, were the 
ones who were receiving comp/Ssnsatory services.- In dollars their 
. services were from 3% to 349% more costly per student than those 
offered non-compensatory students. Schools with compensatory reading 
programs funded by Title I tended to spend slightly more per student in 
absolute amounts (except for one special category of special reading ' 
-group) but slightly less in relative amounts (viz. relative to services ^ 
provided non-compensatory only students) than did schools with compensatory 
reading programs funded from sources other than Title I. 
Oh the average compensentory students tended to be as close- or closer 
to their non-disadvaritaged peers in the spring than in the fall with 
the extent of this "catch-up" being greater at the lower than at the 
higher grades. Although such results do not readily lend themselves 
/to cost-effectiveness calculations* they do suggest that student reading 
skill acquisition and liking for reading are enhanced by. these additional 
•resources^ However, extensive analyses did not uncover .a clear relation- 
ship between the level of resources (or their corresponding dollar amounts) 
and the magnitude of skill growth experienced . Rather, given a minimal 
level of resources, the ways in which they were utilized appeared more 
important .than the sheer amount (e.g» the'use of instructional aides 
for clerical or custodial functions rather than instructional activities).** 

•> See Flynn, et al., 1976b. 

** See Flynn, et al. , 1976b for these extensive analyses. 
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. • ■ ' ' . r . 0 

Earlier results of this study also showed that compensatory students 
were behind their less disadvantaged peers in their level of reading 
skill, development (as indexed by their percentile rank) In. the fall to 
aboiit the same extent ..at each grade level. Such results suggest that 
the benefits students derive^ from these a^ed resources may not 

♦ • _ . } 

accumulate across the years. Of course, this latter conjecture is 
tempered by the fact. that it is the most educationally needy students 
who are to be served each year and they are not necessarily the same 
students from one year to the next (viz» a compensatory project does 
not carry "along its successes). Then too, compensatory students may 
experience greater skill losses over the summer months than do their 
non-disadvantaged peers. 



1 

50 



Our fifth question asks: 

• 5. Were there any unusually effective programs, andif there were, 
what made, them so?* \ 
It m^y be. recalled .from our earlier discussion that 29 schools which 
displayed a range of effectiveness (e.g. high/ medium, and low) ' 
in terms of thei; compensatory students* growth i|i reading skills. during 
the academic year were selected for indepth study. Teams of trained** 
pbseirvers visited each school twice: to verify that the reading programs 
operated as they were believed to on the basis of the teachers* deiscrip- 
.t ions of their activities and to develop * possible explanations for project 
performance; and, to further refine and clarify those explanations. Neither 
the observers nor the school personnel being visited knew the values of .the 
effectiveness ratings . . 
After the visits were rjompleted the effectiveness scores wer&i'compared 

,^th the observers* judgments concerning project performance. There was a 

\ • ' 

consensus that five of the programs were unusually effective by both 

standards. Four of the programs were funded by Title I. The fifth was 

. -A • . • 

a compensatory programs which owed its origins, in part, to Title III of 
ESEA. • \ " 

These five compensatory programs were always either well above average, 
or rrear average''^ their degree of effectiveness. However, their per- 
formance did not necessarily follow a" consistent pattern across the three 
grade levels studied. For example, one school was unusually n 



V ^ ■ . ■ 

* Details on th/t'^e results can be found in Triemeu, et al., 1976a. 

\ . ' • - ' . . ■ .• 

** Their training ^ emphasized what to observe and how to make these 
observations objectively and reliably. 
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effective. at the second grade whereas another was so only at the 
second and sixth grades. Of the remaining three,' two. lacked a sixth 
grade but i^ere unusually effective at the other grade levels" Fhile. ^ 

t.he last schddl was unusually effective At all three grade levels. 

. ' ■ \ ' ' . ■" • 

Although not selected in terms of their children's percentile gain*, the . 

■ ' - . ■■ ' \ * ••" ' • " ■ ••''* . 

'availability of Fall and Spring norms'for the Metropolitan Readying 

scores^ at the fourth\grade allpwed forrsuch a comparison. In the 

■ - • ' ^ • • • . • • 

three programs that were unusually effective at this grade level, 

a typical compensatory student ma:nifested an average percentile. 

• \ . » ■ 

shift of' seven ranks from the Fall to the Spring (from the 17th 



• percentile in the Fall to the '24th percentile in the Spring on the 
Metropolitan total scale). For ;the subtests corresponding results 
were: . * . \ 

■ •■ ' ' - \ * ' 

Fall \ Spring. 

Word Knowledge. ' 17 \ 

\ Reading 17 \ ^27 

When the costs of the compensatory programs in jthese f ive tschools ' 

were compared with those in the remaining schools>.they were not coii- 

sistently more or less expensive. Rather, on the average they cost 

about the- same.^ the average of the other schools.**" 



* Unusually effective compensa>ory"program6 were identified by the 
extent to which the average of . all their test scores in the S pring 
exceeded that which' could be expected on the basis, of their Fall 

. scores (the residual gain criterion). . See Trismen , et al., 1976a 
for these analyses. Percentile ranks were computed in the same 
manner as described earlier • 

** See Flynn, et. aljrl976b. ' 
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Thete were Borne elements that the observers felt:, were cpinmon to • ' 
these five programs. First, all had defined reading as an important 
instructional goal, had assigned it priority among the school " 
activities and had manifested, this commitment by expending more 
time on reading or on having a better quality of reading ... 
resources. Second, in all five schools, there was effective educa- 
tional leadership specific to the issue of ;reading\ (in three of five • ' 
, it was the principal). Third, an outstanding feature of all five 
was careful attention to basic sfcills. Fourth, in 'all* of ' these ■ 
schools- there was a relative breadth of materials.- Last, in all five 
schools there was evidence of cross fertilization of ideas among , 
teachers. ''^ . ■ "* 

Additional- insights into-'the nature of effectiveness were gained 
from detailed analyses of the observations^ made by the teams when' they ^ i 

' ■ . Kpt 

visited the 29. schools.* Those classrooms on which there was achievement 
effectiveness information from the prior year ywere visited by 'two 
observers each time, at approximately eight Week intervals, to rate classroom 
activities and student-teacher in^leractioii. These ratings were found • ' 
to contain eleven .charactleriBtics which differentiated among the 
"classrooms of these 29 schools. When these characteristics were 
relat:ed to the achievement effectiveness information the following, 
results -were obtained: 



* See Triomen, et. al, 1976b, pp. 1,-19': 



In grade 2, effectiveness was significantly and positively - ^ 

* . ■ ■ ■• ' 

related to the degree of adult-centeredness of the class- 
room, to teacher Vf feet (liking for the students) and to the 

level of joint involvement of student-teacher interact-ion,- in learning, 

\ ^\ ■ '' ' * 

Effectiveness was negatively related to equality ' 

J of teacher attention ^to\8tudehts (second grade. only). . 

\ \ ' ■' • ■ ■■ o 

In grade 4, effectiveness was again\positiv8ly related to the 

. ■■ ' ■ \ - ' x • • • . . • . 

degree 'of adult centeredness of the classroom and negatively 



related to punitive conti^l by the teacher. 

In grade 6, effectiveness was positively related to the 

degree of student autonomy and to the equality of teacher 

attention to students. 

Effectiveness was negatively related to classroom 
. ■ . ■ • ° 

affect and teacher warmth (sixth grade on\y) .. ..^ 
* '■ "• 

These results e^^iggeet that different kinds of * teacher behaviors 

may be effective depeadlhg* upon the age/grade/maturational level 

of the students involved;^. Such results may also Kelp to explain, 

in part, the fact that some of the unusually effective programs 

were not uniform in. their effect across the grade levels. 



Our sixth question is really a set of queD'cions similar tc the 
preceding , only focusing on summer prograns^."' The first question 
asks: "What was the incidence of summer compensatory programs, 
their nature and coats?" 

Roughly 26 percent of the^ 266 schools participating in the study 
during the regular school year of f ered some^ type of compensatory^ 
activities during the summer months. AlmostXhalf of the schools 
that had sununer programs (41%) were included for further study 
on the^basis of their willingness to participateand their having 

a sufficient nuciber of summer students to' make such an examination ' 
possible.* When these latter schools'- (viz. the 27; summer program 
schools) were compared to the others, they were found to be located - 
more frequently in the suburbs, to have a compensatory program 
funded by Title I during the regular school year, to have slightly 
greater concentrations of compensatory students during the regular 
school year, and, to a' lesser extent had more experienced teachers 
who expressed greater satisfaction with their administration**^ 



* . The information on which these die?cussions are based can be found in 
Tables 24 th^tiugh 45-of Trismen, et ;al., 1976b and Al-Salam, et al. , 
1976 for the. cost results,. Schools included for the summer study 
tended to have higher achievement scores during the regular school 
year than did the; remaining schools / Schools that refused to 
. participate in' the summer study would have raised the achievement 
scores of summer students slightly had they been included whereas 
those eliminated because of too few'students would have lowered the 
summer average somewhat had they been included. . 

** See Tables 2A and 25 of Trismen, et^ al. , 1976b for these revolts. 
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Summer programs differed from regular school year programs In the 

> . ■ ' ■ _ . 

following ways:* . - 

. ■ ^ ' " -a. 

. .. Attendance was voluntary . 

All but one^f ' the schools had a shorter school- day. 

School staff indicated that their instructional groups 

were smaller , ' that their instruction was morp. flexible 

. and individualized, that they were more concerned with 

remediation and enrichment and that they used a greateri 

variety of .instructional materials. 

... Smaller class sizes resulted in a ^eater percent of 

tota'(. summer program cost going to staff expenditures 

(B8p in the summer versus 69% in the regular, year) . 

Costs per student hour of Instructicm wercjovar 2.5 times greater 

i' r in/ the summer than during the regular school year (56c per 

summer session hour versus 22c per regular school session hour). 

/ / .•. Per student hourly cost in schools that received 

Title I funds were 1.7 and 1.6 times more 

expensive for the regular .'md- summer sessions \ 

* ■ . \ 

' ■ . ' \ 

respectively, than in. schools that did^ not receive - \ 

\ 

\ such funds. 

•J \ .■ \ 



* Se.e pages 3QA34 of Triamen, et al.-,' 1976b ■ and^Table^ 22-24 of . 
Al-Saiim, 1976 for these results and those in the next sebtion. 
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Summer programs differed from one another In the following 
ways:. 

,. . Severity-eight percent of the schools had a five or six 
week summer program, 18% had a four week program and 
4% an eight week program. 

The most frequent bases for determining pupil partlcl-. 
patlon were: depressed reading levels (24%); teacher 
or staff recommendations (24%); and, parent request 
(21%). ■ ' . 

..In 85% of all summer study schools summer compensatory 
reading programs were funded wholly or in part by funds 
supplementary to the regular school budget. 



\ 



..• Forty-six percent of all summer study schools used 
Title I funds for such purposes. 
The most frequent instructional approach was a combination ■ 
of linguistic-phonetic and language experience (63%) 
followed closely by linguistic-phonetic alone (11%) and 
ecletic (11%). 

... Title I funded programs differed from others 
in that they spent more time; improving motor 
abilities xelatedX^ reading. 
Cost differences among schools were pronounced and 
were primarily due to dlf-ferent^itilized resources, 
different program lengths and dlf fer^nj:. student-staff 
ratios. 
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Cost differences among grade levels were not 
appreciably different being 56, 54 and 46 
dollars per student for grades 2, 4 and 6 
•respectively. 
• • Programs that had a remedial/competxsatory emphasis 
were about twice as expensive as those that had an 
enrichment or enrichment/remedial emphasis (about 

$31 per student for the latter types versus about 

■ ■ • ■ ■ • ■ \ ' , ^ 

$67 to $75 per student for the former types). 

< ■ • ■ ■. 

Title I funded programs were about 77 percent more expensive " 
than those without such funding ($70 per student for Title I 
versus $40 for the others, this difference being due primarily 
to the greater compensatory emphasis of the former).* 
Our second question cdncernihg summer compensatory reading programs asks 
"How do students who attend them. differ with regard to their test scores 
and background?"** 

. Students who attended a summer program differed from regular 
year students in their same s^chools in the following ways: 

In the Fall students who were to attend summer compensatory 
programs the subsequent dummer attained slightly higher 
overall test, scores than did other compensatory students 
who would not attend such a program. By Spring their 
(viz. summer attendees) test scores substantially exceeded 
those of regular year compensatory students except at the 
second grade where they ^ had fallen slightly behind the 
other- compensatory studeuts. . . v , 
"* See Tables 16 and 17 of Al-Salam, et al., 1976. 

** See" Tables 31 through 34 of Trismen, et al., 1976b. These analyses 
pertain to. students who attended u 1973 summer program. During the 
1972-73 school year they were in the second, fourth or sixth grade. 



. ... These same patterns also prevailed when 

summer attendees were compared to regular 
year compensatory students in schools that 
did not offer ^ummer programs. " 
In their liking for reading, summer attendees were no 
different from their regular year compensatory counter- 
parts for both the fall and spring of the second grade. 
However, at the fourth grade summer students were 
slightly more favorable for both; fall and ^spring results 
while at the sixth grade they were less favorable in the 
fall and more favorable in the spring. . V : ; 
When compared to compensatory students in schools that \ 
did not, have siimmer programs, summer attendees tended 
to have more positive attitudes toward themselves ^as 

: \ 

readers. . . M 

■\ \ 

Eighty-one percent of summer students'^were white whereas 
during the regular school year only 63% of the compensatory 
students in schools,.that had summer programs were white.' 
Fifty-three percent were males whereas during the 
regular school year 57% bf the compensatory students 
in schools that had summer programs were males. 
Of the summer 'school students, 35% participated in the 
free lunch program duringWihe regular school year. 
During the regular school' y^ar about 50% of the students 
in schools that had summer programs participated in the 
free lunch program. ■ ' 
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• . About 48% of the summer students had had prior assistance 

in compensatory reading whereas during the regular 
school year almost 62% of compensatory students In 
these schools had received some form of prior com- 
pensatory assistance in reading, 
• Students who attended a summer .program differed from 'one 
another in the following ways: 

• • Students in. Title I funded schools scored lower than 

students,, in nbn-^Title. Iv^^ Spring 
... .tests and on those administered at , the completion of 

their summer session. 
These results suggest that studiints who attend stimmer compensatory 
programs are proportionately somewhat less educationally and 
economically disadvantaged (as indexed by their test scores and free 
lunch/minority status, repectively) than are regular year compensatory 
students who do not fetttend .^uch sessions. Further^, students in 
Title I funded. summer prbgrams are more educationally disadvantaged 
(as indexed by their test* scores) than are other summer attendee^ . 

Our last question about summer progtaius asks: "How did students 
benefit from their attendance and, if some programs were more effective 
than others,. what accounted for their success?"^ 



* See Tables 39-43 of Trismen, et al., 1976b. The reader is reminded 
that these analyses are based on only 27 schools involving less than 
200 students. 
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• . Students who attended summer programs were able to ' 

maintain the level of skill development they had 
demonstrated on the Spring tests (viz. they did not ' 
display, losses over the summer months as would be ^ 
expected of students who did •not;..atte.nd such programs, 
especially disadvantaged -students). 

. . Students in Title I funded programs achieved an • 

amount that was equal to • that of students in other ' 

programs. 

... As a consequence the reading skill differences 
among these categories of students remained 

• ; . the same from the spring to end of the Hummer 

• session (viz. Title I students did not fall 
fall further behind). " 
. Some programs were more successful than others- [(viz. their 
Students achieved to a greater or lesser extent than would 
be expected on the basis of their Spring test scores*].'/ 
When successful programs were compared with unsuccessful 
ones it was. found that the succrssful ones: 

. . Concentrated on grade 2 programs and less on mul^i- 

.■ ■ • ■ ' 

age programs. 

.. Had more teachers from other" school districts and 
fewer who taught in that same, school during the \ ■ " 
regular school year/ 



There were six such schools for students who had completed grade 2 the 

prior year (three successful and three unsuccessful. Similarly, there 

were four such schools for students vlic had completeci grade four the 
prior year (two successful and two unsuccessful). 
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.. Had more' experienced teachers, who indicated greater 
satisfaction with various aspects of ^ the program, 
and tended to disagree in greater number, to the 
following:. "The pupils want to learn^but: they don't 
have the, r^ght background . for school work"/ 
. o. Did. not differ frpm the others' in terms^ of their type of 
approach, or .their level of resources utilized or the 
level of associated cost of these resources (viz, they 
•were neither more nor less expensive tha^^ the others).** 
The seventh and final question askR: ^ \ 

: . ; . ■ . . \ ■ 

"How do the results of this study compare with those from earlier time 
periods and other Title I evaluation studies?". ^ 
Th±B is the first comprehensive national stydy of compensatory reading 
programs, most. of. which were funded by Title I of the Elementary and 
Secondary Education Act (nearly 58% were Title I funded) . Early national 
evaluation studies of Title I were inconclusive due in part i to the infancy 

■ ■ . ■ ' ■ . ' • ■ ' ■ V • 

and diffuseniBss of the program (not targeted on basic skills, and not ' 

' - . \,. . 

serving the most needy t idants) and due also to the lack of 'availability 

/■ ' . . ■ ■ . . \ ^ ' 

of evaluative data (early national evaluations depended upon picking up 

■ . . • . - ' \ ■ . ■ ■■ 

whatever data happened to be available locally***). However, la trend observed 



* See page 61 of Trlsmen, et al., 1976b. 
** See Chapter 5 of Al-Salam, 1976. 



*** A practice which, even now, will not permit methodologically! sound 
inferences to be made about program impact. (See for example 
"Assessment >of Reading Activities. Funded Under the Fecleral Program 
of Aid for Educationally Deprived Children", Government Fringing 
Office, Wa«Hington, D.C., Decwnber 12; 1975). ' ' - 
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In the annual State Title I reports which have become more complete 
in retent years, tends, to show that student participants achieve at 
a rate that.is equal to or greater than that of the average student, 
vhile they, are in the program*. 

Further support for these results comes from local Title I evaluations 
which show an increasing incidence of highly successful projects, (by 
highly successful is meant that project participants narrow the 
distance they are behind their more advantaged peers by about one- 
third or more)**l The results from these three sources of evidence^, 
(viz. national, state and local Title I evaluations) contrast with . 
those 'from .a national study conducted just prior to funding of Title 
I which showed that disadvantaged (minority) students fell increasingly 
further behind their more advantaged (non-minority) peers as they 
progressed . through their years of schooling .and developed an increasing 
sense of fatalism about their ability to improve their life chances 
through the avenues of education***. 



See Gamel, et al., 1975; and Thomas, et al., 1976. . 

** See:. Annual Evaluation Report on Programs Administered by the U.S. 
■ Office. of Education, FY 1975, pp. 91-9A. U.S. Department of, Health 
Education and Welfare, Office of Education, Office of Planning, 
Budgeting, and Evaluation. (See Appendix D of this Summary for the 
relevant pageb) . 

*** See Coleman, et. al, 1966; May eske, et.al, ^1973a , L973b, 1975 and for 

more recent eviderice, Ozenne, 1973. Older students of a given baclcground 
were further behind majority students than were their younger counter- 
. parts of ^t he sanje background. 
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Prior- to 1965 there was less of a national focus the achievement 
performance of disadvantaged children and, as a consequence, their per- 
formance was seldom selected out for comparison with their non-' 

» .. . 

disadvantaged peers. However, as part of this study, an historical 

overview of students* performance on standardized readin tests was 
made. The survey concluded that for the ^rty year period prior to 
1965 the average s.tudent ^(elementary and^econdary grades) showed 
a progressive increase in his or her reading skills^ However, at 
about 1965 this 'increasing trend slowed down and since, then a very 
Slight decline may even have set in*. . This latter conjecture tends to 
be supported by other evidence. For example, jresults from, college 
entrance examinations administered near the completion of high school, 
show a definite decline in the performance of* prospective applicants 
in the verbal and mathematics areas over this same time period. 
Similarly, results from the National Assessment of Educational Progress ' 
show a' decline in Science and Writing skills during a. five year peribd>*. 



All of these trends pertain to ^the average or abwve average student. 

What then might characterize the status of disadvantaged students 

for this same time period?^ Since most compensatory activities are 

concentrated on the early elementary grades and focus on basic skills 

it is instructive to note from the National Assessment results for Writing 



See: Farr, Tuinman,and Rowls (1974) 

See: NAEP Newsletters of 1975 , (VIII, . (2)) and 1976 (IX(1)) 
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that nine year olds actually improved in theirVriting skills 
when compared, with their 'counterparts' of four years earlier. 
Similarly, a special study of functional literacy ."^^also conducted ' 
■by National Assessment, showed that 17 year olds in 1974 performed 
better than their counterparts in 1971 and that those who showed: 
the greatest gains were children of parents with low educational 
backgrounds.* These two trends of a slowing down in the rate of 
improvement for the typical or average student and ai. acceleration of 
this rate for disadvantaged students suggests that compensatory 
education may be wdrking against a general cultural trend.. The\ 

fa'ctors underlying th^s cultural trend, if indeed one can 
be said to exist at all, are unknown at this time.** 



\ ■ 



* See: NAEP Functibnal Literacy U975) ■ ^- 

** See especially .Hamischf eger and Wiley (1975) for an examination of 
different bases for a trend and for an explication of the diffe^rent 
factors inyolved. Investigation of a similar trend in England for 
this, same time period suggested that their decline- was due to two 
. factors: (1) a growing tendency on the part of early elementary teachers 
to let students progress at their own pace rather than adhering' to 
f lx«d standards of accomplishment for given age/grade groups coupled 
• with; (2) a lack of recognition on the part of later elementary teachers 
that they wer? responsible- for basic reading instruction and a lack 
of preparedness to engage in such instruction (Start, 1972; Start 
find Wells, 1972), Reeults from National Assessment on changes in • ' 
reading skills over a four to five year period will become available 
In-mid 1976, and will be more definitive concerning the existence 
of .such a trend. 
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Conclusions i 

The preceding results say a lot about compensatory reading programs 
In general, as well as about how the statements might vary .dep^hdin^ 
Upon the source of funds used to support such programs. As before 
they can be organized around a set of questions; however , the questions - 
•are. fewer, in number and more, ^general In nature. ^ . 

!• In ^hools that offer compensatory^ programs, are the most / 
educationally needy students being provided compensatory assistance 
In reading? Using the level of a student's performance on a . standardized 
reading measure as an index of his need for assistance the answer is 

.1 ■ ... ■ •■. ^ • 

unequivocally yes. Schools that receive Title I funds have greater 

concentrations of low scoring students and provide assistance to more 

I ■ ' . 

of them than do non"-Title ll schools. 

Since the incidence of such needs differs depending upon a student's 
membership in various subp'opulat ions, proportionately more of these i 

•-»•••'■• I ! * 

subpbpulatlons are provided services. For example, jproportlonately'' 

^. ■■■ I - ■ . . 

more older than younger students, more boys than girls, more poverty 

than non-poverty students (using free lunch participation as the index) 

and more minority tMu non-minority students reclBlve services than one 

would expect, merely on the basis .of their Incidence in the general " . 

population and these latter 1 two are_especlally so for Title I schools. 

Nevertheless, the provision of services appears to be guided mainly by 

their need for special aissi^tance in reading. 



2. Are the compensatory services supplemental to those usually 
provided students'^ in regular (non-compensatory) reading programs? Using 
cost, as developed -from the resource-^ost model, as an index of the level 
of resources provided, comptosatory students do receive more services 
than their non-compensatory counterparts, and the level of additional 
resources varies with the nature of their ihsiructional group. mem'b^hip. 
Title I schoolisv tended to spend more than noij-Title- 1 schools in^' absolute ' 
amounts, but non-Title I schools tended to spend more in relative amounts 

(viz. they spent slightly more on their c.ompenjsatory students compared to 

• ' ■ ' • ■' . ' ) ■ 

non-compensatory students) when^they did *offer such special assistance. 

However, the incid^ence of small, special reading groups (the most expensive 

kind of instrucfe^Lonal group) was much greater, for Title I schools. ' 

-3. How do compensatory reading students benefit from their special 

assistance? , ^ . - 

The nature' of the results .tended to vary' somewhat depending upon the 

subtest Rnd. grade level being examined and upon the analytic technique 

employed.' All of th^e^^agig^ the assertion that c6mpensato.|ry 

students tend not" tofall further behind non-compensatory students during 

the academic year (the , ma in exception \;as for a 'scale called Word Knowledge) 

Also,, percentile analyses showed that with re'gard to the SOtb percentile 

(national norm) student, both compensatory and non-compensatory students" 

Wnlfested improvement ev»^n though the latter had 'a somewhat greater 

percentile •'ncrease than the former. Otlier analyses^, suggested -^that for- 



some of the' sub-tiests , compensatory students , were closer to non- 

*■ 

compensatory students in their reading skill performance in the 
Spring than In the, Fall.. However, since compensatory students 
tended.^to.. rank at about the same percentile in the Fall of the 
second, fourth and sixth grades it was sixggasted that gains acquired 
during the academic year tended not to hold up in subsequent years.* 
Such results would be affected by the fall off over the summer 
months and the fact that the programs tend to serve the tad st needy 
students each year and th^y are not necessarily the same students 
from one year to the next (this is especially so for Title I) . 

Students who-receive compensatory assistance in reading tended to 
become favorable towards themselves as readers^anci toward thefr 
reading activities to a degree =. that was /equal to or greater than that 
of their unassisted peers. This was so to the same extent for.Title I 
as for non-Title I students. 

■ . . ■ -.1 ■ ■ ■ ' 



This obsery^.uion is not based on a folIov~u|> .of the same individual" • 
students over a period" of years'. ^ ^ ^ 



4., How do unusually effective ^programs differ from the others? 

' ' ■ ' ' i ' ■ • ,. ' ■ ' :■ 

The five unusually effective programs could not be differentiated analytically 
from the others in terms of their use of a single approach to compensatory 
reading. Similarly, their cost was neither more nor less than that of the 
other programs. The five appeared to share a number of common elements 
concerned with what one might term a set of "planning, and management 
variables" (viz. all^manif ested a commitment to rfeading with a careful 
focus on basic skills guided by effective leadership, with ideas shared 
among staff members and with the staff having access to a relatively bro^d^ 

array of materials). The same programs were not -uniform in their degree' 

^ ' . • . ' ■ 1 ■ .... ' \ . . 

of effectiveness at the '.different grade levels — an observation for which 

^ ' . i ■ . ' '\ ... " \ 

other data suggested that a teacher^s way of relating to his/her students 

*~ I. 

might be involved. An index in ^percentile ranks^of the extent to which 
jinusually effective programs exceeded the others was available from 

the results at the fourth grade • In Word Knowledge the typical compensatory 

'■■\ 

student slipped by two percentile ratiks.from the Fall to the Spring while " 
the typical student in the unusua:ily effective programs increased by seven 
percentile ranks. For the Reading sub-scale^ the typical compensatory student 
increased by four percentile ranks while his counterpart in the unusually, 
effective programs increased by ten percentile ranks from the Fall to 
the Spring. S^uch figures help* to, develop an appreciation of what is 

realistically achievable in the aggregate for compensatory programs— * 
an increase in the seven to ten percentile range in terms of national 
norms'^for inaiviclual students is unusually good. For some programs, 
depending, upon the past .gains of their students served, the prevention 
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of a I086 of this order would be judged unusually good even though, 



as a result 



of such an attainment, their students would only malnt/ialn 

■•V 

their same percentile rank from Fall to Spring. 

Fovir of these five unusually effective programs were funded by/xitle 
I. The fifth had its. origins In party In Title 7 II of ESEA-y 



Questions sjLmllar to the preceding can also be posed of suiter programs. 
Answers to them showed the following: 



1. With regard to need: students who received compensatory 




assistance in a. summer program had depressed reading scores during the 



regular school year althouglx they were not as low scoring as r-egular 
year compensatory students who /did not attend summer programs nor were 
summer attendees proportionately as poor or non-white /as were their regular 

year compensatory counterparty. / - 

funded programs fiad lower test scores 
non-Title I programs < 

of resources: costs p^er student hbur of 



.. Students in Title I 
thian did .students it 
2. With regard to level' 



instruction were 2.5 times greater in the summer than during the regular 
school year. 

" i , 

Title jl funded progranis w'l^re about l.g time's more expensive than 
non-Tiltle I programs. 
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3.- With regard. to benefit: at the time of completion of their 
suimnfer programs student attendees rflld not fall. below the level of skill 



development they had attained In the Spring. 

• • Students In Title I funded programs achieved an amount equal'.... 
to that of students In non-Tltle I programs even though the 1 
former scored lower on the Spring tests than did the latter ; 
(viz. Title I students did not fall further behind). , 

4. ' with regard to unusually effective summer programa: five could b 

identified*,' however, they did not differ from others in terms of their. 

t> ■ ■ 

type of approach nor were thejr more nor less expensive than others — rathe 
a few characteristics of the teaching staff related to their experience 
and attitude toward their students appeared to play a prominent role. 



* Three at grade two and. two at grade four. 
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. r . APPENDIX A . - . 

The purpose of this^. appendix is to illustrate , that very different 
conclusiqns aibout progr.un impact can be~reached depending upon 
whether raw scores or grade level equivalents are used for. analysis 
purposes*. 

The" first set- of figures, given- in Table- 1, compare j the' total number 
of reading skill items, comprleted correctly in the Fall and in the 
Spring. Examination of these data ^shows that compensatory studjents 
(CR) exhibit a gieater increment in the number of items correct from 
the Fall to the Spring than do [either "non-compensatory students in 
their own "schools (NCR) or students iri schools that don't have 

ompensatory rfeading programs.: These differences are more pronounced 
at the second grade than at the higher -grafle levels. Further, when 

V ' ' -■ 

these results are compared with the Metropolitan Achievement Test 
(MA\t) norms at the fourth grade (the only' grade level for which both . 



Fall\ and Sprin^; nojms w;ere available) one^inds that all students 
achieve at a greater rate than does the typicsj. dr average student. / ' 

The aye rage student shows an improvement: of 8' items during the course 

» . . ' ■ .' ■■ ■ 

of the^ year , moving from 63 Items correct in the Fall. to 71 correct. , 
in the Spring. Hence, in terms of raw /scores, compensatory students 
are" closer to their non-compensatory peers in the Spring thaii^in the 
Fall--on the average they tend to catch up although a considerable* 
differen<^e still remains. 



* The data for these analyses are tajLen from Table's 23A and 23B of 
" Trismen, et al., 1975a (see MAT TOTALiS) . 
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Table 1 - Mat Total Number of Items Correct 



Test 



NCR-CR 



NCR ScKopls^ 



Grade Level Administration CR^ NCR** Difference NCR Schools ***GR Difference 



Second 



Fall 
Spring 
• Difference 



AO. 3 56.7 
59;6 70.0 
19.3 13.3 



16.4 
10.4 



53.3 

67.8' 

14.3 



13.0 
8.2 



Fourth 



Fall ' -■! 

Spring 

Difference 



39.3 60.7 
50.0 70.1 
10^7 9.4 



21.4 
20.1 



56.5 
66.4 
9.9 



17.2 
16.4 



Sixth 



Fall 



Difference 



57.8 76.2 
64.2 79.7 
6.4 , 3.5 ' 



18.4 
15.5 



74.3 
77.5 
3.2 



16.5 
13.3 



'6. 



- * CR - students who received compensatory assistance In reading. 

**\ NCR - students In schools wlUrxompensatbxy reading programs who did' not 

• ' receive such assslstance. ^ ./ v 

*** NCR schools - students In schools that do not have compensatory./ reading 
programs. N 



8i 
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Table 2* - Mat- Total Grade Level EqutTalents 



\ 



"Tpst ^ ; NCR-CR 

Grade/Level Administration . CR** NCR** Difference 



Second 



Fall 



.. \ Spring 
Difference 



1.7'6 V2.31 



2.46 
.70 



3.21 
.90 



.55 
.75 



NCR Schools- 
NCR Schools*** CR Difference 



2.21 
3.05 
.84 



.45. 
.59 



Fourth 



Sixth 



Tall 

■ Spring 

Difference 

Fall 
• Spring 
Difference 



2.93 4;36 
3.57 5.32 



.64 



-58 



.96 



4.22 "6.20 
.4.80 6.83 



.63 



1.43 
'1.75 



1.98 



1-2; 03' 



4.10 
5.00 
.90 

6.03 
6.59 
■ . .5-6 



1.17 
1.43 



■i.8r 

1.79 



* CR - stud^ntB who received compensatory assistance in reading. 

** . NCR - students in schools with compensatory reading programs who did not" 
receive such assistance/ ' 

*** NCR Schools - students in schools that do not have compensatory reading 
programs* , 
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Analyses of these data, after they have been converted to grade-level 

— ■ ■■ ■ / ■ ■ •• , . . . / 

equivalents, are given in Table 2. Examination of these results would 

lead one to the concijusion that compensatory students* (CR) are one-, 

half to almost two years behind NCR students in the fall and that 

they fall progressively further behind, during the regular- school 

year with this drop being most severe at the fourth grade and least 

1 " ■ . ■ |;- ■ 

severe at the sixth grade. Such results would 'lead one to the view^ 

that in spite of their assistance, compensatory students:, do not 

achieve at a rate equal to or greater than that of t>Ui±r nisssisted 

peers'. ' In fact one would concl{ide that non-compensator> students (NCR) 

attain almost a full year of growth over this period of time whereas 

CR students attain only six-tenths of that amount. 

■ ' - ■ ■ ' . '< . 

-These results for grade level equivalents are an.iyrtifact of a pro- 
cedure which both^orces test scores to take on. certain properties (by 
making them pass through or^near average performance at different: 
grades) and accentuates small differences in test performance so that 
they assume unwarranted importance (e.g. for some tests at some grade 
levels, one item correct can be worth one-half a year of , growth) . Grade 



-lev el- equivalents do not accurately reflect test results and should 
never be used in educational evaluations. " 
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APPENDIX B 



Standard Score Differences for Compensatory versij^s Non-Compensatory 
Differences by Test and Grade Level j 



• / 

/■ 



Test 



Grade! Level : 



Fall Spring 




Compensatory Students Compared With 
Students in Schpols Yhfit Do Not Have 
Compensatory Programs 



Fall \Spring : Fall Spring 



Metropolitan Word \ 
Knowledge " \ 


-.66 


-.51 


; -.77 . 


V 


-.80 


-.70 


Metropolitan Reading \ 


-.68 


-.59 


-.76. 


-178 


-.82 


-.74 


Metrp,politan Total . 






-.80 


-.80 


-.84 


-.74 


^COOP/STEP II. 


\ -.74.- 

1 


'-.64 • J 

/ 
/ 


-.74 


"-V 

-" \ 


-.86 


• -.81 




■J 






\ 







'Compensatory /students Compared With Non- 
compensatory ^Students from Scholpls that 
Offer Compensatory Programs 



Test 



Grade Level 



Metropolitan Word 



Fall Spring Fall Spring 1 Fall Spring 



Knowledge : . * 


-.84 


-.63 


* -.98 . 


-.98 


-.89 


-.83 


Metropolitan Reading • 


-.86 


■r 

-.74 


-.93 


-.79 


-•91 

\ • 
-.94 


-.83 


Metropolitan Tot^l" 


-.91 


-.74 


-.99 


-.98 


-.87 


COOP/STEP II. 

1 . ■ • 




- . /■ 
-.84 - 


-.95 


-.89 


-.96 , 


-.92 



These data were computed from the values in Revised Tabioii 23A and B of 
Trismen et al.^ i976c by subtJ^acting the means of the two groups ajnd 
dividing by the' sample 'standard deviation. 



APPENDIX C 



Fourth Grade Metropolitan Standard Score Values for Compensatory and Non- 
compensatory Students Taken from the National Fall and Sprliig Means 



Test 

Grade 4* 

Metropolitan 
yjoxd Knowledge 

Metropolitan 
Reading 

Metropolitan 
Total 



Compensatory 

Students 

Fall Spring 



Non-Compensatory Students 

in Schools that offer Non-Compensatory 
Compensatory Programs Schools 



\ 



Fall 



-1.09 -1.03 \ -.11 

\ 



- .97 . - .82 
--1.10 -1.03 



.04 



-.32 



Spring 
-.04 

+.12 

c 

-.04 



Fall 



-.32 



-.21 



-.30 



Spring 
-.23 - 

: -.04 . 
-.23 




* These values were excerpted from Revised Table 23A and B of Trlsmen, et ai., 
. 1976c. ' ' . . 
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appendix' d 

. Evidence of Effectiveness From State and Local Reports 

Another form of information concerning the aggregate benefits of 

Title 1 comes from the annual State evaluation reports. Early in 

FY '73 legislative activities suggested that Title I would retain 

c . ... 

its identify even if consolidation were to occur. Therefore, a 

study was initiated to see what could be learned from a critical 

examination of the information in_r^centLSXate--Tl-t-le I reports 

(FY's 71-74), how such results might have changed when compared 

with earlier years (FY's 69-70 in Wargo, et. al, 1972) and, how 

Stat^ reporting systems might b^ improved.* Results from the first 

phase. of this study, which is concerned with the review of current 

. and past reports, reveals that most continue to show a number of~ 

serious shortcomings which precludes their usefulnes's in making 

statements about the achievement "benefits, of project participants 

at the state level. Most reports do not contain statistically* 

representative data and the data which are presented are almost .always 

expressed in terms of grade level equivalent gains. The data are. 

unrepresentatiye because many LEAjb do not get their reports in on 

time to be used in the State's report and of those* that do, the jdata 

are of ten incomplete and nonrepresentative.** Hence, in. preparing his 

* Specific steps that are being taken* to improve State and local 
. project evaluation practices and reports are discussed in the 
final portion of this report. 

** Some: States have used the Anchor test results ^^to equate achieve- 
ment test scores for grades 4, 5 and 6 (1'974) . However', this 
practice is limited and will diminish as more manufacturers 
revise their tests. 




report the State evaluator is forced to rely only on the available data 
and this is a biased subset of all LEA projects and their participants.* 
Almost all of the States report their achievement benefits in grade, 
equivalent gains— a metric that capitalizes on systematic biases 
introduced by practices of test manufacturers, as discussed in a 
subsequent section. 

Despite these drawbacks some trends across this six-year period could 
be discerned. They were: (1) the numbers of Title I participants showed 
a progressive decrease while expenditures over time showed a corresponding 
increase with the result that average Title I per-pupil expenditures 
increased; (2) most participants were Involved in Title I during the. 
regular school term, most were in the priinary grades and most were 
involved in reading or language arta programs; (3) expenditure data which 
were available I'sl^owed a substantial and continuing increase for instruc- 
tion and a decrease for construction and equipment; (4) there was a heavy 
emphasis on direct educational services in contrast to services supportive 
of the instructional program with reading and language arts receiving 
highest, priority; '(5) needs assessment information indicated that 
reading and mathematics are the most frequently identified areas of 
need end that st-andardized tests are used to determine student needs; 
-'(6)''^f or the small number of states for which Impact data were fo^und 
to be valid (about 17) student participants manifested growth equivalent 
to or greater than the national average; however, their fall ^t.est scores 

* The direction of the bias is probably positive if one recognizes that 
children present at the beginning and end of the school year are 
likely to be more academically able than tho/je who leave. 
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at successive grade levels showed that such gains as did occur v/ere 
not cumulative across the years » undoubtedly for some of the same 
reasons cited earlier (summer losses and serving the most needy each 
year) as well. as due to the States use of Grade Level Equivalent scores 
for reporting gains (Gamel, et al. , 1975). 

Ir a recent search for effective reading projects sponsotcsd by the 
Right-to-Read program (viz. the search was not limited to compensatory 
projects) some 1500 candidates were iu;;r:lfled. Of this total about 52 
percent eliminated themselves from considszva on (by failing to respond 
to the survey que.^* . cnuaire) / Of the ,72R if . ng only 27 (or less 
than four percent) ver found to meet defOTA5ti:l'£ vt^indards for claims 
of effectiveness (e.g.^ a-Jequate criteriosi roeasur&s^ statistical adequacy, 
experimental design, etc.). Of these 27 proj-act.??, OE*s Diosamination and 
Review Panel (DRP) approved 12 as meeting adequate evalua?:ion standards 
(this -represents a survival rate of less than one percent of 1500 or about 
l.e percent of the 728) Of those that were approved by the DRP eight 
were compensatory projects and four of these were funded by Title I 
(Bowers -J et. al, 1974). Such results show that the problems of adequate 
evaluation procisdures are nci limited to a particular Federally funded 
program but are rather endemic to the educational sector. 
Tliese re8uli:a can be contrasted with i:hose from a survey conducted by 
th.fi Title J program staff. In this survey each State was- encouraged 
to^nominate two effective projects. Fifty-one wero received, screened 

andy reduced to 28 by the OE i'taff . Thes^; 28 we::e then site visited to 

■\ . " 

makev detailed observations of them and to insure that: ":;^y were in com- 

\ • • ^ ' : ■ •. ■ . -. ■ 

pliance with regulations. The 17 survivors from this latter screening 

\ . ' ■ ,' 
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stage were submitted to the DRP; 11 were approved for dissemination.. On 
the basis, of these two studies (as well as^ ti\e foregoing) it can be 
asserted that the evaluation requirements for^Title I "lead the way" 
for the evaluation of State r id locally funded ''projects. Indeed^one— - 
might question, whether effectiveness concerns would have attained any- 
where near the prominence they have during the past decade were it not 

; \ • ■• 

for the Title I evaluation requirements, \ 
A third, earlier search conducted by pPBE, sought to identify, validate 
and package up to 8 effective approaches to compensatory education so' 
that schools in other locales could duplicate the projjrictsir by workiag 
directly fron the package (Tallmadge, October 197A) . isome 200 projects 
were considered as potential candidates for packaging. Initial screening 
on three criteria reduced this number to 136. The three 'criteria vers 

. that the program had to; emphasize reading and math benefits; be 
oriented toward disadvantaged chllSren; and, be evaluated more than 
once. Of the 136 surv:Ivors, flr-itail6d descriptive Information could 
obtained on only 103. Fifty-four percent of these were rejected due 
to inadequate evidence of effectiveness as determined by an exceptionally 
rigcrous examination which included independent analyses of project. raw 
data and on-site visitations. Hence, six projects were selected and their 
specific implementation req^uirements were packaged in whal: have come to 
be called "Project Information Packages" (PIP's) (five of these six. 

>were Title I funded). These six packages are now being field tested 



\ 

\ 
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to see If results In other sites can be produced which, are comparable 
to those of their original site.* 

•CO ■ 

When the effectiveness data for the above projects were belrg carefully 
va lidated (Tallinad ge_and^orst-«— 1-97A-)— s ome-heret'ofore unrecognized 
effects of the practices of test manufacturers were revealed. Since 
these effects are dramatic In nature and have profound implications 
for. the conduct of all evaluations t^ey will be dealt with in some 
detail here.t. " . » 

Many test manufacturers obtain their "norm"^ata (namely, data on how 
a nationally representative sample of students perform on the test) 
during the middle of the academic year, about February. For many pur- 
poses including program evaluation, however, norms are desired so that 
one can gauge their students* standing relative to other students at, 
the beginning and at the end of the school year. To fulfill this need 
the manufacturers usually create "synthetic" norms by drawing a smoothed 
curve through the average or median scores for consecutive grade levels. 
This curve is then assumed to represent the growth throughout the 
academic year for a typical or average student. HoweVer, students 
probably do riot grow according to this kind of a curve. They may 
forget a great, deal over the summer and may learn more during some 
periods of the year than others. Consequently, this smoothing pro- 
cediire introduces systematic biases which can produce some of the 
following results depending upon the grade level involved: (a) project 
students can show better than month for. month, gains* yet never catch up 

* For more details on the nature of the field test see the evaluation 
projects described under the 'Packaging and Field Testing Program. 
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wlth their more advantaged, peers; (b) project students are virtually 
precluded from showing month for month gains or better since the 



typical or averagelatudent— onlygains two-thirds of a month per. month. 



In addition, some test '--ublishers break the uine month academic year up 

■ . ■. ■ .... . fl- 

into three equal segmt^nts. For example, starting with September 1st ^ 

I • . ' • - ■ 

as the beginning of the school year, three months of growth would occur 

.■■ i- ■ ' ^ ■' ■ ^ ■ 

between November 30th and December 1st and another three months of growth 
would occur between February 28 and March .1st. As a result of these 

^ kinds of synthetic norms, a- program that administers its pre-test late 
in the Fall and then jpost-tests early in the Spring will show more 
month per month growth than a program that tests early in the Fall and 
late in the Spring, even though the latter program might.be considerably 
more effective than the former. Finally, the use of grade equivalent 
scoires, rather than standard scores or percentiles, was shptjn^to 
systematically distort the amount of growth even vh4n real norms were 
available for the time period under consideration. As a result projects 
can be judged effective and worthy of dissemination when they aren't 

. and project participants can be judged as catching up with their more 

privileged counterparts when they aren't. Or alternatively, on occasion 

I- 

effective projects can be rejected as being ineffective. The antidote to 
all this is to use only those testis which have real norms appropriate for 
the time interval under ifstudy and to base the evaluation on standard scores 
' and express the results in percentile, ranks,* These results hve profound 
.implications for the upgrading of State and local Title I evaluation 
practices discussed in subsequent paragraphs. . ' ' 



* For example, participating students moved from the I2th percentile on 
the pre-test to the 33rd percentile on the post-test. 
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